The development of high quality multimedia devices, such as set-top boxes, high-end televisions, digital televisions, personal televisions, storage products, personal digital assistants (PDAs), wireless internet devices, etc., is leading to a variety of architectures and to more openness towards new features for these devices. The development of these new products and their ability to display video data in any format has resulted in new requirements and opportunities with respect to video processing and video enhancement algorithms. Most of these devices receive and/or store video in the MPEG-2 format. In the future many of these devices may also receive and/or store video in the MPEG-4 format. The picture quality of these MPEG sources can vary between very good and extremely bad.
Next generation storage devices, such as the blue laser based Digital Video Recorder, will have high definition (HD) capability to some extent. A Digital Video Recorder (DVR) is a good example of a type of device for which a new method of video image enhancement would be advantageous. An HD program is typically broadcast at twenty million bits per second (20 Mb/s) and encoded according to the MPEG-2 video standard. The storage capacity of a Digital Video Recorder is in the range of approximately twenty to twenty five Gigabytes (20 GB to 25 GB). This represents about two (2) hours of recording time of HD video per video disc.
To increase the amount of record time per video disc, long play modes can be defined. For example, in a Long Play (LP) mode, a broadcast bitrate of twenty million bits per second (20 Mb/s) may be recorded at a storage bitrate of ten million bits per second (10 Mb/s). This will provide about four (4) hours of recording time per video disc. In an Extended Long Play (ELP) mode, a broadcast bitrate of twenty million bits per second (20 Mb/s) may be recorded at a storage bitrate of five million bits per second (5 Mb/s). This will provide about eight (8) hours of recording time per video disc.
The process of transforming a high bitrate bitstream to a lower bitrate bitstream is referred to as “bit rate transcoding.” One method of transcoding a high bitrate bitstream into a lower bitrate bitstream involves the steps of decoding the high rate bitstream with an MPEG-2 decoder and then encoding the resulting bitstream at the lower bitrate. Another method of transcoding a high bitrate bitstream involves directly transcoding the bitstream to a lower bitrate without fully decoding and re-encoding the video. This method is known as Direct Bitrate Transcoding (DBT).
The process of MPEG-2 transcoding may decrease the picture quality (e.g., sharpness) of the video sequence due to the loss of information. However, it is desirable that the picture quality not be compromised too much. This is especially important for the Long Play (LP) mode. Therefore, the post-processing of transcoded video signals plays an important role in improving the perceived picture quality.
Most of the state of the art sharpness enhancement algorithms have been developed and optimized for analog video transmission standards like NTSC, PAL and SECAM. Traditionally, image enhancement algorithms either reduce certain unwanted aspects in a picture (e.g., noise reduction) or improve certain desired characteristics of an image (e.g., sharpness enhancement). For the newly emerging digital storage devices, digital televisions, set top boxes, and other similar devices, the traditional sharpness enhancement algorithms may perform sub-optimally on MPEG encoded or transcoded video due to the different characteristics of these sources. In a closed video processing chain of a storage system, information that allows the quality of the encoded source to be determined can be derived from the MPEG stream. This information can potentially be used to increase the performance of video enhancement algorithms.
Because image quality will remain a distinguishing factor for high-end video products, new approaches for performing image enhancement, specifically adapted for use with these digital sources, will be beneficial. In a paper entitled “A Compressed Video Enhancement Algorithm” by C. J. Tsai, P. Karunaratne, N. P. Galatsanos and A. K. Katsaggelos, Proc. of IEEE, ICIP '99, Kobe, Japan, Oct. 25–28, 1999, the authors propose an iterative algorithm for enhancing video sequences that are encoded at low bitrates. For MPEG sources the degradation of the picture quality originates mostly from the quantization function. Thus, the iterative gradient-projection algorithm employed by the authors uses coding information such as quantization step size, macroblock types and forward motion vectors in its cost function. The algorithm shows promising results for low bitrate video. However, its main disadvantage is its high computational complexity.
In a paper entitled “Improved Decoding of MPEG-2 Coded Video”by B. Martins and S. Forchammer, Proc. of IBC2000, pp. 109–115, Amsterdam, The Netherlands, Sep. 7–12, 2000, the authors describe a new concept for improving the decoding of MPEG-2 coded video. Specifically, a unified approach for deinterlacing and format conversion, integrated in the decoding process, is proposed. The technique results in considerably higher picture quality than that obtained by ordinary decoding. However, to date, its computational complexity prevents its implementation in consumer applications.
Both papers describe video enhancement algorithms using MPEG coding information. However, both of these scenarios, in addition to being impractical, combine the enhancement and the cost function. A cost function determines how much, and at which locations in a picture, enhancement can be applied. The problem that results from this combination of cost and enhancement functions is that only one algorithm can be used with the cost function.
It would therefore be desirable to have an apparatus and method for video enhancement capable of effectively enhancing encoded and transcoded video sources.